Maximum Entropy Density Estimation with Incomplete Data

We propose a natural generalization of Regularized Maximum Entropy Density Estimation (maxent) to handle input data with unknown values. While standard approaches to handling missing data usually involve estimating the actual unknown values, then using the estimated, complete data as input, our method avoids the two-step process and handles unknown values directly in the maximum entropy formulation. The maxent method was recently proposed as an excellent method of presence-only prediction [2, 3]. In a presence-only framework, we are given a set, X , of data in which some of the data are labeled as positive. However, unlike the typical classification framework, the remaining unlabeled instances are not necessarily negative. Instead, they are considered of unknown class. The regularized maxent method treats the positively labeled points as random draws from some hidden distribution overX and attempts to estimate that distribution. Specifically, regularized maxent tries to find a distribution over X with maximum entropy such that the expected values of each feature are close to the observed means of the features with a positive label. Let F be anN×D matrix of features such that Fij is the i’th datum’s j’th feature. Let vectorm be the means of the D features of the labeled positive data. Then the standard regularized maxent optimization is: